Next: 5 References
Up: Appnotes Index
Previous:3 Technology Description
The most significant testability information supplied by DM was :
Dependency modeling is particularly useful in gaining insight into system testability very early in the design process, even at the concept phase, as soon as the functionality of the system has been defined. Figure 4-1 illustrates the concept of a Functional Block Diagram. and it also can continue to make significant contributions to the system development through architecture selection. A special feature of DM is that the model remains completely adaptable to the level of system information available at the time of the testability examination. More than one modeling exercise is likely as the system design continues to evolve.
During concept development, time- and cost -independent dependency modeling can provide a definition of ambiguity groups resulting from imposition of test point restrictions and can give a very early indication of testability problems. As the system becomes better defined, test time and test cost can be factored into a dependency model if the designer can supply estimated or actual values for component mean time between failure (MTBF) and mean time to repair (MTTR). It is also possible for the user to prioritize known component function criticality. The DM recommended test strategy will be biased by the relative values of these three factors.
For candidate architectures, during architecture tradeoffs and selection, quantitative trade-offs based on:
Collected requirements were grouped into two classes :
1. |
Create a set of requirement templates for the 3 project phases: design, manufacturing, and field support |
2. |
Check the inter-phase inherited anomalies for consistency and completeness: design flaws escaping to manufacturing , design flaws escaping to field, and manufacturing faults escaping to field |
3. |
Within each project phase, look for opportunities to merge the identified flaws/faults within the specific phases. Consider test means and available support as first level bases of comparison. Document differences, if any, in quantitative requirements for requirements which are merged. |
4. |
Scan through all test phases for common test means at detection and isolation coverage levels. Do not consider the inherited faults in this comparison. Formulate a preliminary core Singular Test Philosophy (STP). Also do not consider correction coverage since, in most cases, the test means will be manual procedures only. |
5. |
Consolidate the merged requirements formed in Step 3 across the 3 test phases with the intent of applying the preliminary STP to all phases, if possible. Several possibilities exist: |
• the STP applies to the 3 sets of preliminary merged requirements |
|
• the STP can be modified by addition, deletion, or reordering during negotiations among representatives of design, manufacturing, and field |
|
• individual templates can be modified to force conformance to the STP |
|
• individual templates can be tagged as not compatible with the preliminary STP |
|
6. |
Examine applicability of the STP to the inherited requirements. Negotiate an agreement on the STP by any of the means identified in Step 5. |
7. |
Perform final consolidation by negotiation among the 3 phases: |
• choose a final STP |
|
• add phase-specific supplemental items to the STP as required |
|
• modify individual templates to conform to the STP as required |
|
8. |
Document the STP along with any exceptions and attach the documentation to the Consolidated Requirements Document. |
9. |
Sign off the Consolidated Requirements Document. |
A test strategy is a specification for an ordered application of a series of test means which will lead to defined levels of coverage for detection, isolation, and correction of specified fault classes. Coverage specifications are quantified and measurement techniques are specified in detail in the requirements templates. RASSP DFT seeks a test strategy called a singular test philosophy (STP) which is a strategy that can be applied during design, manufacturing, and field deployment. It is a desired goal because it allows reuse of test stimuli developed during simulation in manufacturing and in the field. The need for specialty tests is minimized and significant costs associated with development of phase specific test program sets (TPS) are eliminated or greatly reduced.
For BM3, common sets of test means to cover the phase-specific faults for each project phase were relatively easy to extract. A pattern of BIST followed by Boundary Scan test followed by less automated testing means was developed in each case. Negotiation sessions among the phase experts then examined the test means series extracted from the design, manufacturing and field environments with the intent of discovering possible application of a test means series across all phases. Examination again showed that a preferred arrangement existed - BIST, Boundary Scan (BS) Automated Test Equipment (ATE), non-BS ATE, and manual test. In some cases this basic series needed to be supplemented by means such as inspection or simulation. As will be discussed in more detail in the steps presented in the following example, an STP did exist for BM3.An STP did exist for BM3.
An example of using the individual steps identified in Table 1 and applied to BM3 will now be discussed. The cost-benefit analysis of applying these steps to a project can be found later in this Application Note in Section 4-4, DFT Economic Analysis.
Step 1.
An annotated template establishing the requirement for "Detection coverage for design flaws" along with annotations which describe the individual entries is shown below. It is typical of the templates for all project phases.
Requirement Name : Detection coverage for design flaws
Realizable ______ Consistent ______ Valid ______
a. Test Phase
b. Test Means
These are the selected ordered set of test means available for detecting design flaws. Simulation is the primary method at the modeling level. Other means apply to the design phase up to and including production of the first prototype hardware. As will be seen in Step 7 where test requirements are finally consolidated, the last four test last means belong to the STP. Simulation is a supplemental test means appropriate to the design phase. These test means are assigned independently by the expert representing the particular project phase, in this case Design.
The design phase is a non-operating test.
d. Degree of Allowable External Support
No limitation is made since any available technician and/or engineering support will be made available to support the design effort.
e. Fault Model Assumption
The listed set of design flaws are typical of a board design project.
f. Quantitative Definition of Metric
This ratio defines the effectiveness of the design process.
g. Prediction and Validation Weighting Factors
Previous experience with board design processes establishes a likely distribution of similar flaws on future projects. These factors represent the capability of a specific company with a specific type of project.
h. Quantitative Requirement
This number must approach 100% for a reasonable design process. Some tolerance must also be allotted to reflect the potential for an imperfect process.
i. Allowable Requirement Compliance Tracking Methodologies
Prediction of design flaws is based on analysis of the circuit function and the typical rate of discovery of flaws for a circuit of that type. Fewer flaws per unit time will be uncovered as the design approaches its completion.
Simulation is the primary design analysis tool. Lack of design flaw effects during simulation establishes the capability of a design. The rate of discovery during simulation can be compared with the predicted values for a consistency check.
The ultimate means of compliance tracking is actual measurement of performance of a design. A design performing as expected indicates the absence of design flaws.
Step 2.
Templates showing coverage for inherited anomalies were added to ensure that no design flaw escapes (non-detection) from design to manufacturing and field phases or manufacturing faults from the manufacturing to field phase were accounted for. This coverage is added to reflect such possibilities and their effects upon satisfaction of the requirements. The possibility of inherited anomalies generally reduces the levels of detection, isolation, and correction which can be achieved by any real physical system.
Step 3.
An effort was made by each of the project-phase-specific representatives to examine the extent to which flaws/faults (titles of templates) in their respective areas could be "merged" so that the number of templates was reduced. This was accomplished by searching the requirements templates for commonality among test means, support requirements, test mode, and quantitative requirements for the baseline set of flaws/faults established by that expert. This within-phase flaw/fault consolidation was performed independently for each of the phases of design, manufacturing, and field support. As flaw/fault consolidations occurred, the fault model entry (template entry e.) would be modified to reflect the extension of coverage. and Individual quantitative requirements were explicitly specified as needed to document the post-merged results and preserve the established quantitative requirement values. Consolidation never eliminates important established requirements but may combine them. For BM3, an example of a fault merge is the template for "Detection coverage for manufacturing faults - Bridging, Open, and Stuck-At".
Step 4.
Following the within-phase flaw/fault consolidation, the test means for the identified coverages for detection and isolation within each test phase were examined for commonality with the intent of establishing a candidate local (within-phase) singular test philosophy (STP). For BM3 the test means subseries : BIST, Boundary Scan ATE, and standard ATE were appropriate. BIST and Boundary Scan-based The first two testing means were very cost effective from the life cycle point of view and their tests were executed in the shortest times. These tests are introduced during early simulations and, once developed, accompany the product through the manufacturing and field deployment phases. The need for specialized manufacturing or field testing is reduced or eliminated.
Additional coverage needed in any phase for a specific flaw/fault could be provided by supplementing the core test means set with flaw-/fault-specific test means. Inherited flaws/faults were not considered in the search for an STP to prevent forced commonalities from having an excessive biasing influence (feedforward or feedback) on any candidate STP. Correction is not included as a comparison category, since correction is almost always a manual process.
At the end of this step, the requirements have been merged by test means within their respective life cycle phases. Three phase-specific candidate STPs exist.
Step 5.
Examination of the sets of test means applicable to the individual phases will exhibit the likelihood of obtaining a true STP across all phases at this point. Commonality will lead to a merging of the sets of test means across the phases. Lack of commonality will indicate that a singular test philosophy may not exist. In most cases, the extent of an STP derived at this point will be a core STP which may need supplementation within individual life cycle phases. Results of the attempt to generate an STP may be any of the following cases :
The first two cases represent situations where an STP can be readily defined. For BM3, the second case applied. In all but a few instances, the original assigned ordering of in-phase test means in the templates was very similar and it was relatively easy to reorder the individual test means sequence for a specific nonconforming flaw/fault to bring their final ordering into conformance with the majority of the test sequences. Thus, the basis for a core STP was found.
In a few cases, the core set of test means was supplemented with appropriate means. In particular, INSPECTION was added as the first test means for manufacturing to consider the reality that a manual or assisted inspection should precede execution of BIST or any subsequent means when physical faults such as shorts or opens are the faults of interest. SIMULATION was added as a supplemental test means for design.
Step 6.
The STP was next applied to the inherited anomalies. For BM3, no significant problems were encountered and the STP was readily applied to the anomalies. Some reordering was required in a manner similar to that used in the across-phase consolidation.
For BM3, little effort was required in this area.
Step 7.
Final consolidation was achieved by taking the cross-phase STP and adding the supplemental test means as required.
For system test, the test means arrangement is : SIMULATION, BIST, BSCAN ATE, NON-BSCAN ATE, MANUAL
For manufacturing test, the test means arrangement is : INSPECTION, BIST, BSCAN ATE, NON-BSCAN ATE, MANUAL
For field test, appropriate test means sets are :
MISSION OP is added as a supplemental test means and corresponds to observation of the continued operational status of the equipment during mission operation.
In this case, the basic sequence is BIST, MANUAL to conform to the requirement that the equipment execute BIST during field deployment.
This arrangement is essentially the same as that for the manufacturing test since field Depot test corresponds to return to the manufacturing source for evaluation/repair.
The core STP is evident in all cases :
BIST, BSCAN ATE, NON-BSCAN ATE, MANUAL
Steps 8. and 9.
The STP was documented for BM3 by generation of a Consolidated Requirements document.
For BM3, requirement/prediction TSDs for selected flaws and faults during design, manufacturing, and field deployment were produced for the FE as a subsystem and for the FE divided into its three PC board (one FPCTL, two FPCAP) constituents. Other TSDs could have been generated, but the handling of these fault classes were well understood, so and standard fault management control techniques would be applied. TSDs are used to their best advantage to specify/monitor coverage for faults not fully understood.
BIST - detection 10%, isolation 20%, correction 0% - Values for this test means are low since the FPCTL board contains only 2 components (of 15) with an incompletely understood BIST capability. A reasonable assumption for is to assign the BIST values based upon the percentage of circuitry which contains BIST. Correction receives a 0% value since BIST will not change conditions causing a short fault.
BOUNDARY-SCAN ATE - detection 40%, isolation 40 %, correction 0% - BS-ATE is assigned a relatively high value for detection since BS-ATE can detect short faults as well as faulty components in the interconnection paths. The actual values are chosen by examination of the interconnection routing and an estimate of the percentage of the board circuitry accessible to BS-ATE testing. Isolation is given the same value as detection since BS-ATE testing is intended to locate faults by proper choice of test vectors. BS-ATE does not perform correction, thus the 0% value for this transfer function.
NON-BS-ATE - detection 50%, isolation 50%, correction 50% - Non-BS ATE is testing on an external tester. The detection level possibility here is related to the number of traces/components which are accessible via a test connection interface such as bed-of-nails. Isolation for non-BS ATE testing is assumed to be the same as for detection. Correction is given a value of 0% for this test means for FPCTL.
MANUAL TEST - detection 50%, isolation 50%, correction 100% - Detection and isolation are assigned estimated values based upon the nature of the testing anticipated. Values assigned reflect the ability of a "typical" manual test to identify and locate a specific fault causing the problem reported by a previous test means. Manual test is expected to be used mainly as a path to correction. It is not intended to be a primary means for detection of faults because the time consumed by a manual test as a primary test means for detection and isolation is long and such a test is costly. Correction is assigned a value of 100% - otherwise the board under test is declared non-functional.
Similar lines of reasoning were used to assign other transfer functions used in TSDs. The resulting values were assumption-based. Their use will allowed reasonable allocation of expected values for detection, isolation, and correction events. The assumptions used can be modified to conform to a particular company's capability to execute the testing required. For example, if a company does not have non-BS ATE, it cannot execute such a test and either the available means must perform more effectively or other additional test means must be made available to replace the "missing" capability.
Test cost entries for TSD attributes come from estimate of the fixed and variable costs associated with testing. Fixed costs are based on the cost of test equipment needed to test all boards produced, fixture costs and nonrecurring labor cost for test development. Variable test costs are associated with technician labor required to perform tests. The amount of labor required is directly related to the test time discussed in the previous paragraph.
Three worksheets form a set capable of controlling/monitoring of a specific flaw/fault. A separate workbook is assigned to each flaw and fault for which requirements have been established. Individual worksheets are assigned to requirements, predictions, verification, and measurement related to the flaw/fault. For any one complete workbook, the number of worksheets will be up to 12. At system design levels, fewer sheets will be needed since verifications and/or measurements may not apply.
During the system concept and design development phases of a project, the requirements and prediction worksheets are populated. Verification and measurement worksheets will be populated as the project proceeds into the actual design and fabrication phases.
Figure 4-4 identifies the contents on the worksheets for one TSD class (requirements, prediction, verifications, or measurements) in one EXCEL workbook. For the sake of analysis, a Rrequirements TSD set will be discussed.
Sheet 1 contains a flaw/fault specific TSD which states the requirements for testing related to that flaw/fault at some level of packaging. On this sheet, the user will enters the transfer functions, cost and time allocations appropriate to the flaw/fault being processed.
Sheet 2 contains a roll-up requirements TSD for a specific flaw/fault and the allocation of these requirements among the contributing lower packaging levels. For example, a TSD describing subsystem requirements will have board components. Downward allocation ensures that each lower level packaging level is examined to ensure that it will satisfy requirements which must be met in order that the requirements of the higher assembly be satisfied. The roll-up is the topmost section of Sheet 2. The roll-up TSD is automatically created from summing the contributions from the individual lower level TSDs.
TSDs at different packaging levels are coupled by EXCEL intersheet and inter-workbook communication. Thus the roll-up TSD from a lower level TSD can function as a complete Sheet 1 entity at some higher packaging level.
Sheet 3 contains the differences between the requirements stated on Sheet 1 and the roll-up section of Sheet 2. Sheet 3 entries are automatically generated by intersheet subtraction. During system design phases, reported differences between requirements and predictions could suggest that there could be a problem with the requirements.
Similar sets of TSDs for the same flaw/faultcan be constructed to specify reporting means for verifications and measurements. The TSD set thus serves as the common connection link among requirements, predictions, verifications, and measurements and specifies values for detection, isolation, and correction for each of these categories.
Also, any testing requires access to the physical testers required to perform the tests. If expensive ATE is currently not available, either it must be acquired or the test strategy calling for its use must be modified. For any company, a one-time analysis of facility test equipment in terms of expected project needs should be performed to ensure that the test architecture required to support test plans is appropriate. The analysis for BM3 generated a list of test means available to Lockheed Martin to evaluate different potential equipment anomalies. Tests selected for the STP were chosen from this list based upon availability of specific test means.
When all the details of test architecture are resolved, a test plan which orders application of test means in accordance with the TSDs is generated. For the BM3 exercise, BIST and Boundary Scan test were used as primary test means even though the FPCTL board was not built using a majority of BIST or JTAG components. (Customer requirements demanded use of these test means.) BIST and Boundary Scan test were augmented with the use of a wide range of test means at the disposal of the LM facility. Test procedures were developed in template form to provide test personnel with a well defined test sequence and a statement of all required test data.
A primary tool was the VHDL simulator which produces models and testbenches to add and evaluate the effects of DFT features. QuickVHDL and IKOS simulators were used to model BM3 hardware and testbenches. The former is a utility system and the latter can generate code which can execute on a general purpose processor or can be applied at high speed on a hardware accelerator.
Other tools, mostly VHDL-based, are used to install particular DFT features into a design as required or to perform testability analyses.- Automatic Test Pattern Generators (ATPG) such as VICTORY by Teradyne and board level BIST tools by LogicVision are examples which were used with BM3. The purpose of their application in the Shadow Program was to prove that the capability for installation of DFT features existed and was supported with a toolset that made DFT practical.
Another category of DFT tools which falls into the support category and is usable even after the design process has been completed-. The Parallel Port Tester (PPT) by Teradyne, which is used to test hardware during both manufacturing and field support phases is an example. This tool uses a portable PC to apply test vectors, generated during the design phase to equipment in operational environments.
DFT design tools will be used as needed on a RASSP project. The nature of the specific hardware/software design will determine which design tools are applied. Some examples pertinent to BM3 will now be presented in the following paragraphs.
TBC control of multiple circuit blocks was also simulated in a testbench to simulate testing of multiple boards from a central station. The FPCTL board of BM3 can use this mode for testing 2 FPCAP boards, and can, in turn, be tested by a higher level controller. The VHDL system model used a TBC to control three simple virtual boards. Virtual board 1 was simulated as a chain of PRPG, logic block, and PSA as described above. Two other boards were simulated by PRPG/PSA pairs. The testbench for this case sent parallel commands to the TBC which serialized the command in compliance with JTAG protocol and configured one ti8245 as a 16-bit PRPG and its receiving counterpart ti8245 into a PSA. The PRPG then supplies test vectors to the logic and associated PSA. Under control of the testbench, when the random pattern completes, the signature acquired in the PSA is shifted over the JTAG bus to the TBC which converts the serial message to a parallel data word and makes it available for further processing. Virtual boards 2 and 3 were sequentially configured by the testbench via the TBC as PRPGs and returned patterns via the JTAG bus to the TBC to verify how a microprocessor would use the TBC to control and access data from multiple boards.
The VHDL testbenches and model code used for the TBC analyses remain as legacy items for future designs.
As part of its normal activity, VICTORY generates a Test access port (TAP) integrity test, which verifies that the JTAG connection path for the compliant board components can support a JTAG-based interconnection test. Part of this procedure is a verification of the boundary scan description language (BSDL) files for all components. These files which describe the JTAG features of each component in a VHDL-like language are obtained from component manufacturers and are critical to a successful JTAG interconnect test for the components.
Both VTM_TOP and VICTORY were applied to the BM3 FPCTL board, even though the JTAG string was very limited. All features of the tools were tested. The primary result was a demonstration of the accessibility analysis. Outputs from these tools will be significantly more important as the number of JTAG components on a board increase. In the limiting case of 100% JTAG component compliance, the SVF file will be capable of executing a full interconnection test for a board. Both VTM TOP and VICTORY are general purpose tools which offer considerable manufacturing test support and should be considered for use on any RASSP project.
A support tool which works with the SVF file generated by VICTORY is the Teradyne parallel port tester (PPT), a tool which accepts an SVF file and applies the JTAG interconnect test to a board under test via the parallel port of a PC. The PC-based test allows the interconnect testing to be executed during manufacturing and field support using only a PC attached to the board via a 5 wire JTAG test bus cable. In cases where the JTAG-compliant component complement is relatively high, the PC test can replace more elaborate equipment and provide testability which is not normally possible. The testing will take longer because of the serial vectors, but the test accessibility feature is a good trade. For BM3, PPT installed on a portable PC was used to evaluate the use of the 8245 buffer/PRPG/PSA device using VICTORY-generated SVF vectors. PPT was also used to test the DATA I/O board from the RASSP Benchmark 2 project.
If memory components can be tested independently, the need for a software-based processor-controlled test that requires much of the board to be functional before memory testing can begin, is eliminated. A simulation (complete with testbench) of an FPGA which addressed and write/read data to each memory cell of a 256K x 4 static RAM to validate its storage ability was successfully executed. This is a promising new technique which can extend test coverage to complete arrays of components which have traditionally not been independently testable. Another desirable feature of FPGA-based board level BIST is the same testing is available for both manufacturing and field testing. LogicVision has announced a commercial product (memBIST XT) which is capable of installing an FPGA-based memory test capability at board level.
LogicVision also created other board ABIST tools during their participation in the RASSP program which were used to perform "what if" analysis on the BM3 controller board. The tool was used to investigate the fault coverage improvement resulting from replacement of JTAG compliant components on the board in place of those actually used and test coverage effects of strategic JTAG buffer insertion. The BM3 controller board was a good example for analysis but a poor candidate for improvement since only 2 of 15 components had boundary test capability and 2 others had BIST capability. No significant improved coverage could be achieved for the hardware already constructed to meet project timing constraints. Future versions of the hardware could benefit from the analysis, however, by substitution of JTAG-compliant components, where possible, for the current components and the addition of JTAG buffers.
The general conclusion was that DFT inclusion can result in savings in each of the project phases, with DFT-based cost savings increasing as the project life cycle moves from beginning to completion.
The methodology used for determining LCC reduction effects associated with RASSP DFT was to define an assumption-based model derived from experience and outputs from existing DFT application studies. For example, an economic analysis specifically targeted to BM3 quantitatively compared detailed estimated costs for manufacturing quantities of boards tested with BIST/BS and ATE test means. Test costs for each of the project phases before (using conventional test methods) and after application of the RASSP DFT methodology were computed. This approach presents a methodology which can be adapted to particular circumstances by changing the assumptions appropriately. For BM3, analysis began with assumption of a model for the distribution of LCC :
This distribution model reflects historical costs incurred for a "typical1” project requiring fabrication of a large (1000s) number of boards. A three step before/after/compare procedure was applied (see BM3 Shadow Report Summary for basis of assumptions) to estimate the effects of testing on a system with this LCC distribution.
Step 1. Estimate pre-DFT contributions of testing to LCC.
When these quantities are summed, the testing portion of LCC attributable to testing is 17.4%.
Step 2. Impose RASSP DFT methodology and reevaluate effects of testing on LCC.
When these quantities are summed, the portion of LCC attributes-DFT cost of testing is approximately 7.8% of LCC cost.
Step 3. Draw conclusions.
Several significant conclusions can be drawn by a before and after comparison of test costs in steps 1 and 2:
To verify that DFT offers savings throughout the product life cycle Potential DFT savings at each project phase were also examined to show that DFT offers savings throughout the product life cycle. A summary of the application of DFT to each project phase and the DFT-related potential cost differences is summarized in Table 2.
W/O RASSP DFT |
W/RASSP DFT |
Cost Difference Estimate |
Design Phase |
||
System Design |
HW/SW Codesign |
+10% |
Functional Design |
VP Design/Test Analysis |
+10% |
Prototype Fab |
VP Testbench Design |
-10% |
Prototype Test |
VP Simulation Execute |
-10% |
HW/SW Integration |
(Part of Codesign Effort) |
- |
System Debug |
Functional Simulation |
-20% |
Prototype Mods |
VP Model/Testbench Mods |
-20% |
Retest |
Rerun Simulation |
-10% |
|
||
The potential savings (sum of the cost difference estimates) during design resulting from effective DFT installation can be up to 50 %. |
||
Manufacturing Phase |
||
Inspection |
Inspection |
- |
Manufacturing Proto Fab |
Manufacturing Proto Fab |
- |
ATE Setup |
BIST/BScan Setup |
-200% |
Generate Final TPS |
Reuse Design TPS |
-100% |
TPS Validate |
Reuse Design TPS |
- |
Board Test |
Board Test |
-50% |
Retest |
Retest |
-50% |
The potential savings during manufacturing can be up to 400% if effective DFT is built into the product. The main source of savings is in reuse of the TPS generated as part of the design effort. As expected, the manufacturing phase benefits considerably from DFT. |
||
Field Support Phase |
||
TPS Re-engineering |
BIST/BScan Reuse |
-1000% |
Spares Test (Depot) |
Spares Test (Local) |
-100% |
Repair/ATE-based Retest |
Repair/BIST/BScan Retest |
-50% |
The potential differences for field support costs with and without DFT are very large(1150%) due to the known high cost of Test Program Set Re-engineering. |